Estimating causal effects of public health education campaigns using propensity score methodology

Many evaluations of public health education campaigns attempt to draw conclusions regarding the effect of messages on audiences’ attitudes, beliefs, and behaviors based on observational data. To make causal inferences in these instances, it is necessary to adjust estimated campaign effects for possible selection bias due to systematic differences between respondents exposed to the campaign and those that were not. In particular, it is necessary to adjust for the impact of confounding variables that are likely to be determinants of both campaign exposure and outcomes. In comparison to other available methods for adjusting for selection bias such as multiple regression and instrumental variable methods, propensity scores offer a particularly simple way of adjusting estimates of campaign exposure effects for selection bias. This paper discusses the logic of this approach and illustrates its application to the evaluation of the National Youth Anti-Drug Media Campaign. q 2005 Elsevier Ltd. All rights reserved. In 1998, the US Office of National Drug Control Policy (ONDCP) launched the National Youth Anti-Drug Media Campaign (NYAMC) as part of an effort to reduce and prevent drug use among youth nationwide. The campaign has progressed through three phases of increasing complexity and intensity. Phase I (January–June 1998) involved pilot testing the intervention advertisements in 12 metropolitan areas. In Phase II (July 1998 through July 1999), these advertisements appeared in multiple media nationwide, not just in the test areas, largely using pre-existing advertisements. Phase III, which is currently in progress, marks the full implementation of the campaign 0149-7189/$ see front matter q 2005 Elsevier Ltd. All rights reserved. doi:10.1016/j.evalprogplan.2005.01.004 * The data used in this paper were collected under contract with the National Institute on Drug Abuse for the Evaluation of the National Youth Anti-Drug Media Campaign (contract # N01DA-8-5063). The Westat Corporation holds the contract for the evaluation with the University of Pennsylvania as subcontractor. We are grateful to Co-Principal Investigator David Maklan (Westat) and his colleagues who have collaborated closely with us on the project, including on aspects of the study reported here. The views expressed in this paper are those of the authors alone and do not represent those of the evaluation team. * Corresponding author. Tel.: C1 732 932 7500x8123; fax: C1 732 932